2 research outputs found
Improving Resnet-9 Generalization Trained on Small Datasets
This paper presents our proposed approach that won the first prize at the
ICLR competition on Hardware Aware Efficient Training. The challenge is to
achieve the highest possible accuracy in an image classification task in less
than 10 minutes. The training is done on a small dataset of 5000 images picked
randomly from CIFAR-10 dataset. The evaluation is performed by the competition
organizers on a secret dataset with 1000 images of the same size. Our approach
includes applying a series of technique for improving the generalization of
ResNet-9 including: sharpness aware optimization, label smoothing, gradient
centralization, input patch whitening as well as metalearning based training.
Our experiments show that the ResNet-9 can achieve the accuracy of 88% while
trained only on a 10% subset of CIFAR-10 dataset in less than 10 minuet
Jais and Jais-chat: Arabic-Centric Foundation and Instruction-Tuned Open Generative Large Language Models
We introduce Jais and Jais-chat, new state-of-the-art Arabic-centric
foundation and instruction-tuned open generative large language models (LLMs).
The models are based on the GPT-3 decoder-only architecture and are pretrained
on a mixture of Arabic and English texts, including source code in various
programming languages. With 13 billion parameters, they demonstrate better
knowledge and reasoning capabilities in Arabic than any existing open Arabic
and multilingual models by a sizable margin, based on extensive evaluation.
Moreover, the models are competitive in English compared to English-centric
open models of similar size, despite being trained on much less English data.
We provide a detailed description of the training, the tuning, the safety
alignment, and the evaluation of the models. We release two open versions of
the model -- the foundation Jais model, and an instruction-tuned Jais-chat
variant -- with the aim of promoting research on Arabic LLMs. Available at
https://huggingface.co/inception-mbzuai/jais-13b-chatComment: Arabic-centric, foundation model, large-language model, LLM,
generative model, instruction-tuned, Jais, Jais-cha